Model-Based Facial Image Coding

This paper addresses the issue of 3-D motion estimation in model-based facial image coding. A new approach to estimating the motion of the head and the facial expressions is presented and has the following characteristics: 1) An affine nonrigid motion model is set up. The specific knowledge about facial shape and facial expression is formulated by this model in the form of parameters. This affine motion model is especially suitable to such a type of nonrigid motion as facial expressions. 2) Based on the affine model, we present a direct method of estimating the two-view motion parameters. Because this method neither necessitates solving the correspondence problem nor computing optical flow, motion parameters can be simply and reliably recovered. 3) Based on the reasonable assumption that the 3D motion of the face is almost smooth in the time domain, we propose several approaches to predicting the motion of the next frame. In this way, the temporal motion information existing in the image sequence is fully exploited. With a good motion predictor the error arising from the treatment of motion by a linear method will be reduced. 4) Using a 3-D model, the new approach is characterized by a feedback loop connecting computer vision and computer graphics. Embedding the synthesis techniques into the analysis phase greatly improves the performance of motion estimation. Our simulations and experiments with long image sequences of real-world scenes indicate that the method developed in this paper not only greatly reduces computational complexity but also substantially improves estimation accuracy. The synthesized image sequence using the estimated motion parameters, a 3-D model of the face, and a frame of textured image looks very natural.